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CN . Abstract 



This paper investigates a singular stochastic control problem for a multi-dimensional 
regime-switching diffusion process confined in an unbounded domain. The objective is 
to maximize the total expected discounted rewards from exerting the singular control. 
Such a formulation stems from application areas such as optimal harvesting multiple 
species and optimal dividends payments schemes in random environments. With the 
aid of weak dynamic programming principle, we characterize the value function to 
be the unique constrained viscosity solution of a certain system of coupled nonlinear 
quasi-variational inequalities. Several examples are analyzed in details to demonstrate 
■ the main results. 
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1 Introduction 

This paper is concerned with a class of singular stochastic control problems with state con- 
straints. The controlled regime- switching diffusion process X and the singular control process 
Z take values in a convex cone Scl". The control problem has the state process 

X(t)=x+ [ b(X(s),a{s))ds+ [ a{X{s),a{s))dW{s) - Z{t), 
Jo Jo 

where W is a <i-dimensional standard Brownian motion, a is a continuous-time Markov chain 
with a finite state space M. = {!,..., m}, Z = (Z±, . . . , Z n )' is an n-dimensional adapted, 
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nondecreasing, and cadlag stochastic process, and b, a are appropriate measurable functions. 
The income rates fi, % = 1, . . . , n, from exerting the singular control are allowed to be state- 
and regime-dependent. The objective is to maximize the total discounted reward 



E 



Y,fi{X{s),oc{s))dZ % {s) 



'l.D 



where r > is the discounting factor. 

Such singular control problems (in various different settings) have been extensively stud- 
i ed in the literature. A partial list inclu des the monotone fo 



( 11984J )). optimal harvesting problem s (Alvarez and Shepp 



((20111), IWehl 119901 )) etc. See also lHaussmann and Suol (1l995al Jbl) for a general singular 



l ower p r oblems ( Karatzas and Shreve 



{19981), ISong et all (l201lh ). opti- 



mal dividend dist ribution schemes (|Paulsen (j2003h ). portfolio selection management with 
tran saction costs ( 0ksendal and Suleml (120021 )). optimal partially reversible in vestment prob- 
l em ( Guo and Phaml f 2005 ) ) . and hea vy traffic modeling and contro l problems ( Lee and Weerasinghe 



stochastic control problem for a multidimensional Ito diffusion on a fixed time horizon, 
in which the existence of the optimal control and the characterization of the value func- 
tion as the unique viscosity solution of a Hamilton- Jacobi-Bellman equation are established. 
Singular control problems w ith state constraints have drawn considerable interests in re- 



Ataretal 



( 120071 ). IZariphopoulou 



cent y ears; see, for example, lAtar and Budhirajal (120061 1 
( 119921 ). among others. 

Note that most, if not all, of the aforementioned literature on si ngular stochastic controls 
deal with Ito (jump) diffusions. One exception is our recent work Song et ah! ( 2011 ). which 
studies an optimal harvesting problem of a single species living in random environments. Due 
to their capability of modeling complex systems with uncertainty, regime-switching models 
have drawn considerable attention from both researchers and practitioners in recent decades 
in a wi de range of a pplications. Some of s uch examples c an be found in mat hematical fi- 



nance ( iZhangl (I200l[ )). e cosystem modeling (jSlatkinl ( 119781 ) 



manu 



Zhu and Yin 



actu ring systems ( iSethi and Zhang! (119941 )). risk management (lElliott and Siul ( 120101 ) 



(2009 )), stochasti c 



Zhul (l201l[ )). to name just a few. In these systems, both continuous dynamics and dis- 



crete events coexist. In particular, the systems often display qualitative structural changes. 
Regime-switching models turn out to b e qui te versatile in captur ing these inherent random- 
ness. We refer to 



Mao and Yuan 



(120061 ) and lYin and Zhul ( 120101 ) for in-depth investigations 
of regime-switching diffusions. 

This work aims to investigate the singular control problem ( II .ip in the setting of multi- 
dimensional regime-switching diffusion with state constraints. First we recall the notion of 
constrained viscosity solution, illustrated by several simple yet nontrivial examples. Then 
we use the weak dynamic programming principle to show that the value function defined in 
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(12. 6 p is a constrained viscosity solution to the coupled system of quasi- variational inequalities 
(12.1 ip in Theorem 14.51 Finally, we derive a strong comparison result in Theorem 15.31 from 
which we establish the uniqueness of the constrained viscosity soluti o n to ( 12.111). Compared 



with t he classical work on viscosity solution such as ICrandall et al.l ( 119921 ). lYong and Zhou 
(119991 ) and others, the novelty and contribution of this work can be summarized as follows. 
In lieu of a single differential equation studied in the literature, this work deals with a 
coupled system of nonlinear second-order differential equations with gradient constraints. 
The coupling effect is due to the presence of random environments or regime switching. This 
feature at one hand makes our model more appealing in real-world applications since it can 
naturally capture the qualitative structural changes of the systems; on the other hand, it 
adds much difficulty in the analysis. In particular, the function F defined in 12.101 is no t 



proper with respect to the variable £ in the sense of the User's Guide ICrandall et al.l ( 119921 ). 



Note th at the properness was an essential assumption in the proof of strong comparison 
result in ICrandall et al.l (119921 ) . Here we need to carefully handle the coupling effect; see the 
proof of Theorem (I5.3P for more details. Another noteworthy feature of this work is that 
we introduce an exponential transformation which allows us to handle both the gradient 
constraints as well as the polynomial growth condition on an unbounded domain for the 
solution of the coupled system of quasi- variational inequalities (12.111) . 

The rest of the paper is arranged as follows. Section [2] presents the precise formulation 
of the problem, followed by some preliminary results in Section [3j We recall the notion 
of constrained viscosity solution in Section HI followed by several examples for illustration. 
Further, in Section HI we establish the existence by showing that the value function V defined 
in ( 12. 61) is a constrained viscosity solution of (12.11ft . The strong comparison result is arranged 
in Section [H A hierarchical PDE characterization of the boundary behavior is arranged in 
Section [5] as well. The paper is concluded with conclusions and remarks in Section El 

To facilitate later presentation, we introduce some notations that will be used often in 
later sections. We say that a function from [0, oo) to some Polish space E is cadlag if it is 
right continuous and has left limits in E on [0, oo). When E = ~R n and £ is cadlag, then we 
write A£(i) = £(t) —£(£—) for t > 0. As a convention, we set A£(0) = £(0). Throughout the 
paper, we use x'y or x ■ y interchangablly to denote the inner product of vectors x and y. 
For any vectors x, y G W 1 , x < y means Xi < yi for every i — 1, 2, . . . , n. The space ofnxn 
symmetric matrices is denoted by S n and the family of positive definite symmetric matrices 
is denoted by <S+. If A, B G S n and A - B G S+, then we write A > B. If </> : R n — >■ R 
is sufficiently smooth, then D Xi (j) = j^, D XiX .<f) = g§-^-, and D(f> = (D Xl <fi, D Xn (j))' is 
the gradient of <p while D 2 cf) = (D XiX .<fi) denotes the Heissian of <fi. For any real- valued 
function /, we use and /* to denote the lower- and upper-semicontinuous envelopes of /, 
respectively. If B is a set, we use B° and Is to denote the interior and indicator function 



3 



of B, respectively. Throughout the paper, we adopt the conventions that sup0 = — oo and 
inf = +00. 



2 Formulation 

We consider singular control problems for a regime-switching diffusion 

d((t) = b(({t),a{t))dt + a(((t),a{t))dW{t), £(0) = x, a{0) = a, (2.1) 

where x G M. n ,a G Ai — {l,...,m}, W is a (^-dimensional standard Brownian motion, 
b : R™ x M h-> K", cr : R™ x M h-> M nxd , and «(•) G .M is a continuous-time Markov chain 
that is independent of the Brownian motion W and is generated by Q = (g^) G ]R mxm : 

m r / .x /x / n -\ I At + of At), if 7 ^ z . 

P{a(t + At)=j|«(t)=z,a( S ), S <t} = r y 1 ; . J / . (2.2) 

[1 + quAt + o(At), if j=i, 

where > for i, j = 1, . . . , m with j ^ % and qa = — Ylij+i % < ® f° r eac h * — 1> • • • > m - 

Throughout the paper, we assume that the coefficients b and a and the generator Q are 
such that for any initial condition (x,a) G lR n x Ai, the solution ( x,a to (12. II) exists and 
is weakly unique. Sufficient condition for existence and uniqueness for stochastic d i fferen - 
tial equation s with regime switching can be found in, for example, iMao and Yuan (hood ). 



Yin and Zhul (l2010h . 

We now introduce singular control into (12.11) with state constraint and suppose that the 
controlled dynamic is given by 

dX(t) = b(X(t), a(t))dt + a(X(t), a(t))dW(t) - dZ(t), (2.3) 

with initial conditions 

X(0-)=xeS, a(0) = aeM, (2.4) 

where Z G M™ is a singular control process to be specified below. Without loss of generality, 
we take S = M" = {x G lR n : X, > 0, i = 1, . . . ,n}. Note that X(0) may not be equal to 
X(0— ) due to an instantaneous push Z(0) at time 0. Denote the solution to (12. 3p with 
initial condition specified by (12 .4p by X x ' a (-). 

Let A X}Q denote the collection of all admissible controls with initial conditions given by 
(12. 4ft . where Z G A x , a satisfies 

(i) for each i — 1, . . . , n, Zi(t) is nonnegative, cadlag and nondecreasing with respect to t, 

(ii) X(t) G S for all t > 0, and 
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(iii) Z(t) is adapted to '■= cr {W(s),a(s),0 < s < t}, where 3o contains all P-null sets. 
Moreover, 

E / e~ rs d\Z\ (s) < oo. 



Note that the state constraint is specified in condition (ii) above. Throughout the paper, we 
assume A XjCe 7^ for every (x, a) E SxM; see Section [3] for a sufficient condition. For a 
fixed Z E A Xt a, the discounted payoff is 



J(x,a,Z):=E / e~ rs f(X x ' a (s-), a(s-)) ■ dZ(s), 



(2.5) 



where / : S X Ai i— >■ M. n with /, representing the state- and regime- dependent instantaneous 
marginal yields accrued from exerting the singular control Zi(t). Assume fi is continuous 
and non- increasing with respect to x in the sense that fi(x, a) > fi(y, a) for each a 6 Ai if 
x < y, where x = (xi, . . . , x n )' and y = (y±, . . . , y n )' satisfy Xj < yj for each j = 1, . . . , n. 
Moreover, we assume < /j(0, a) < oo for each i — 1, . . . , n and a G .M . Such assump tions 
on f are motivat ed by considerations in optimal harvesting problems ( lAlvarezI ( 120001 ) and 
Song et al.l ( 1201 lh ). The goal is to maximize the expected total discounted payoff and find 
an optimal control Z*: 



V(x, a) — J(x, a, Z*) :— sup J(x,a,Z). 



(2.6) 



In order to work with a well-formulated maximization problem, we assume throughout the 
paper that V(x, a) < oo for all (x, a) G M™ x M. 

As usual, we shall rely on the dynamic programming principle (DPP) to deduce the 
behavior of the value function 

' e- rs f(X*> a (s-), a(s-)) ■ dZ{s) + e">V{X*»{n), a(rj)) 



V(x, a) = sup E 



(2.7) 



for every (x, a) G S x Ai and stopping time 77. A heuristic argument using the DPP (12. 7p 
yields that V satisfies the following coupled system of quasi-variational inequalities 

min < (r — C)V(x, a), min {D x .V(x, a) — fi(x, a)} > — 0, (x,a) E S x M, (2.8) 

I i=l,...,n 1 I 

where for any h(-, a) E C 2 , a E Ai, we define 

^ m 

Ch(x, a) = -tr(<rcr / (x, a)D 2 h(x, a)) + b(x, a) ■ Dh(x, a) + q Q jh(x,j). (2.9) 



However, without a priori result on the continuity of the value function, a rigorous proof 
of (12. 7p is nontrivial. Thanks to the state constraint as well as the generality of the set up 



5 



of the problem, it seems not easy to obtain the continuity of the value function V defined in 
(12. 6p . Also, in the current singular control setup with regime-switching diffusion, it appears 
that the DPP is not av ailable from the litera t ure. T o overcome this difficulty, we will instead 
invoke the weak DPP ([Bouchard and Touzil (120111 )); see Section [3] for the precise statement. 
Also, the value function V is not necessarily sufficiently smooth to take first and second 
order partial derivatives. Therefore we aim to show in this work that V satisfies ( 12. 8 j) in the 
weak sense using the notion of viscosity solution. We will show that the value function V is 
the unique viscosity solution to (12. 8p . 

For convenience of later presentations, we define for any (x, a, A) G M n xMx M m x 

R n X S n , 

^ m 

F a (x,^,p,A) = F(x,a,£,p,A) : = r£ a - -tr(aa'(x, a) A) - b(x,a) -p-^qaj^j- (2.10) 



3=1 



Set V(x) = (V(x, 1), . . . , V(x, m)) 1 G M m , then (EH]) can be rewritten as 



min < F a (x, V(x), DV(x, a), D 2 V(x, a)), min {D Xi V(x, a) — fi(x, a)} 

t=l,...,n 



0. 



(2.11) 



for all x G S and each a — 1, ... ,m. 

As we indicated earlier, (12. lip is a coupled system of quasi- variational inequalities. More- 
over, thanks to the term Y^7=x QajCj with Q = (gy) defined in ( I2.2p . for each a G A4, F a is 
not pr oper with respe c t to th e variable £ in the sense of equations (0.1) or (0.2) in the User's 



Guide 



Crandall et al 



1992T). N ote that properness assumption (and in particular equation 



(3.13) in ICrandall et al.l (jl992f )) enabled them to derive the strong comparison result and 
hence the uniqueness of the viscosity solution. Here for our analysis, special care has to be 
given to handle the fact that F a is not proper due to the coupling term. Also, instead of 
working on a bounded domain, we are dealing with unbounded domain S. These features 
make our analysis much more involved than the classical comparison result in 
jl992h . 



Crandall et al. 



3 Some Preliminary Results 

We present some preliminary results in this section. The first one provides a sufficient 
condition for the assumption that A x , a f° r an (x,a) & S X Ai. 

Proposition 3.1. Assume there exists a function $:SxM4 M + satisfying 
(i) for each i — 1, . . . , n and a G M, 

lim ^(x,a) = oo, and lim ^(x, a) = oo, 

|as|— >oo ' Xi\.0 
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(ii) a) G C 2 for each a G M and 

£^(x,a) = b(x,a) ■ D^(x,a) + -tr(aa'(x, a)D 2 ^(x, a)) + y^ j q aj ^(x,j) < 0, 

_ 

j'=i 

for all (x, a) G S x M. Then, denoting by ( x ' a the solution of (12. ip with initial condition 
(x, a), we have 

¥{C' a (t) E S for allt>0} = 1, for any (x,a) G S x M. (3.1) 
Consequently A XtCe ^ /or all (x,a) e S x M. 

Proof. It suffices to prove ( 13. ip . which leads to Z = G A^q for any (i,«)g5xM. To 
this end, we consider (x, a) G 5 x M. and define 

r fc := inf jt > : > Jfe or Cf ,a (t) < ^ for some i = 1, . . . , n 

Note that {rfc} is a nondecreasing sequence of ^-stopping times. Now it is enough to show 
that Tfe — >■ 00 with probability 1. Suppose on the contrary that 

liminf P{r fe < 00} = 5 > 0. (3.2) 

k— >oo 

Applying generalized Ito's formula to the function \I/ and using condition (ii), we obtain that 
for any t > 0, 

E[*{((T h At),a(T k At)] =^{x,a)+E / £q{((s),a(s))ds 

Jo 

< ty(x,a). 

Then since \l/ > 0, it follows from condition (i) and ( 13. 2p that 

a) > E [*(C(r fe A t), a(r fe At)] > E [*(C(r fc ), «(r fc )/ {rfe < t} ] 
> ^ fc P {r fc < t} ->■ 00, as k — > 00, 

where 

\l/fc := inf |\l/(x,j), Ixl = k or Xi = — for some i = 1, . . . ,n, and j & M 

This is a contradiction and hence — >■ 00 with probability 1 as — > 00. □ 
We will need the following proposition in the proof of Theorem 14.51 
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Proposition 3.2. For each aeM and any x,y G S with y < x, we have 

V{x, a) > f(x, a)-{x-y) + V{y, a), (3.3) 
V*(x, a) > f(x, a ).( x -y) + V*(y, a). (3.4) 



Proof. Equation (I3.3p can be established using exactly the same arguments as those in 
Song et all koilh . while ([33]) follows from (J3l| directly. □ 



The next proposition can be established using similar arguments as those in 



(2011) 



Bouchard and Touzi 



Proposition 3.3. Fix (x, a) G 1R" x M. . Then for any stopping time t, we have 



V(x, a) < sup E 
ZeA x , a 



and 



V(x, a) > sup E 
ZeA x ,a 



e- r9 f{X x ' a (s-), a(s-) ■ dZ{s) + e~ rT V* (X x ' a (r), a(r)) 



e- rs f{X x ' a (s-), a(s-) ■ dZ(s) + e^XX^r), a(r)) 



, (3.5) 



(3.6) 



for all upper- semicontinuous functions (p such that V > (p on R™ x J\A.. 

We finish this section with the verification theorem, whose proof is similar to those in 



Song et al. 



{ml) 



Theorem 3.4. Suppose there exists a function <p '■ S x M. h- > M+ solving (12. 8p . 

(a) Then <p(x, a) > V(x, a) for every (x, a) G S x M.. 

(b) Define the non-intervention region 



C = { (x, a) G S x M : max {fi(x, a) — D Xi (j)(x } a)} < > . 

I i=l,...,n I 

Assume there exists a strategy Z G A x , a such that 

(X(t), a(t)) G C for Lebesgue almost all <t < oo, 

/ (D(/)(X(s), a(s)) - f(X(s), a(s))) ■ dZ c {s) = 0, for any0<t<oo, 
Jo 

lim E \e' r{TANAM (j)(X(T A N A N ), a(r A N A /3 N ))] = 0, 

N^oo 

and that if X(s) ^ X(s-), then 

<P(X(s), a(s-)) - <P(X(s-), a(s-)) = -f(X(s-), a(s-)) ■ AZ(s), 



(3.7) 
(3.8) 
(3.9) 

(3.10) 



where N := inf{t > : \X(t)\ > N}, and X = X x > a denotes the solution of (1231) . 
Then (f>(x, a) = V(x, a) for every (x, a) G S x M. and Z is an optimal strategy. 



4 Viscosity Solution: Existence 



This section is devoted to the properties of the value function V. In particular, we aim to 
characterize V as a viscosity solution to the quasi variational inequality ( 12. lip . Let's first 
recall the notion of viscosity solution. 

Definition 4.1. A function u(a;) = (u(x, 1), . . . , u(x, m))' is said to be a viscosity subsolution 
of ( 12.1 ip on S x M, if for any (x Q ,a ) G S x M and functions <£>(■, a) G C 2 (S),a G M 
satisfying (u* — <f)(x, a) < (u* — (p)(x , a ) = for all (x, a) G S x Ai, we have 

min <^ F ao (x , u*(x ), D(p(x , a ), D 2 <p(x , a )), min {^^(20, a ) - /i(x , a )} f < 0. 

I i=l,...,n J 

Similarly, a function u(x) = (w(x, 1), . . . , u(x,m))' is said to be a viscosity supersolution of 
( 12. lip in S x A^, if for any (xo, «o) G 5 x and functions (p(-, a) G C 2 (S), a E M. satisfying 
(u* — <p)(x, a) > (u* — ^)(xo, ao) = for all (x, a) G S x A^, we have 

min <^ F ao (x , u*(x ), ^^(^o, a ), D 2 (p(x , a )), min {^^(xo, a ) ~ fi( x o, «o)} ? > 0. 

I i=l,...,n \ 

The function u is said to be a constrained viscosity solution, if it is both a viscosity subsolution 
in S x M. and a viscosity supersolution in S x A4. 

Before presenting the main result of this section, we shall first study several examples to 
illustrate Definition 14.11 These examples will also help us to motivate later results. 

Example 4.2. Consider the QVI 

min{u(x) - u'(x),u'(x) - 1} = 0, x e (0, oo). (4.1) 

We claim that u(x) = Ke x , K > 1 is a constrained viscosity solution of (14. ip on [0, oo). In 
fact, if x > 0, then we compute 

min {u(x) — u'(x), u'(x) — 1} = min {0, Ke x — 1} = 0. 

Therefore it remains to verify that u(x) = Ke x is a subsolution on [0, oo) using Definition 14.11 
Suppose G C 1 and satisfies (it — <f>)(x) < (u — 0)(O) = for x G [0, oo) in a neighborhood 
of 0. Then it follows that 4>'(0) > u'(0) = K and hence 

min {(f)(0) - 0'(O), 0'(O) - 1} = min {K - 0'(O), 0'(O) - 1} = X - 0'(O) < 0. 

Thus the claim follows. 

Next we show that v(x) = x + 1 is also a constrained viscosity solution on [0, oo). In fact, 
it is easy to see that v(x) = x + 1 solves (14. ip for x > 0. Thus it remains to show that it is 
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also a subsolution on [0, oo). To this end, let ip G C 1 with (v — <f)(x) < (v — (f)(0) = for 
x G [0, oo) in a neighborhood of 0. Then we have <f'{0) > v'(0) = 1 and therefore 

min {ip(Q) - <p'(0), ip'(0) - 1} = min {1 - (p'(0), <p'(0) - 1} = 1 - <p'(0) < 0. 

This shows that v is a subsolution and thus a constrained solution on [0, oo). 

Note that the controlled process corresponding to (14.11) is dX(t) = 1 • dt + • dW (t) — dZ (t) 
or X{t) = x + 1 — Z{t) for t > and the objective is to maximize J(x, Z) = E x J °° e^ t dZ(t). 
For this process, it is clear that A x ^ for all x G [0, oo). Moreover, from the state 
constraint, Z(t) < x + t for allt > 0. Then it follows that 

poo poo poo ps poo 

J(x,Z)=E x / e~ s dsdZ(t) = E X / dZ(t)e~ s ds < E x / e~ s (x + s)ds = x + 1. 
Jo Jt Jo Jo Jo 

Thus the value function V(x) < x+1. In fact, V(x) = x+1 and Z*(t) := xI{ t =o}+I{t>o} Jq sds 
is an optimal control, since J(x, Z*) = x + J °° te-*dt = x + 1. 

To conclude, the value function V is the unique constrained viscosity solution of ( 14.1ft on 
[0, oo) in the class of functions with polynomial growth rate. □ 

Example 4.3. In this example, we demonstrate that the QVI 

min {u(x) — u"(x), u'(x) — 1} = 0, x G (0, oo) (4.2) 

has no constrained viscosity solution on [0, oo). 

First, one can show that u(x) = x + c is not a constrained viscosity solution of ( 14. 21) on 
[0, oo), where c is a constant. Certainly it is the case if c < since for x G (0, — c), we have 
min {u(x) — u"(x),u'(x) — 1} = min {x + c, 1 — 1} = x + c < 0. Now let's consider the case 
when c > 0. The function 0(x) = c + 2x — (1 — |)x 2 satisfies 

[u — <\)){x) = —x + (1 — -)x 2 < (u — (f)(0) = 0, for x > sufficiently small, 

and <p'(x) = 2 — (2 — c)x, 4>"{x) = —2 + c. Thus we have 

min {0(0) - (j)"(0), 0'(O) - 1} = min {c - (-2 + c), 2 - 1} > 0; 

this shows that u(x) = x + c is not a subsolution on [0, oo). 

Next we show that u(x) = c\e x + C2e~ x is not a constrained viscosity solution of ( 14. 21) on 
[0, oo) either, where c%, C2 are constants. Note that for x > small, 

u(x) — Ci(l + x + -x 2 + o{x 2 )) + c 2 (l — x + -x 2 + o{x 2 )) 
= (ci + c 2 ) + (ci - c 2 )x + -(ci + c 2 )x 2 + o(x 2 ). 
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If c\ + C2 > 0, then we consider <p(x) = (ci + C2) + (|ci — C2I + 2)x + |(ci + C2)x 2 . Clearly we 
have (u — <fi){x) < (u — 0)(O) = for x small and 

min {0(0) - 0"(O), 0'(O) - 1} = min |(ci + c 2 ) - ~(d + c 2 ), |ci - c 2 | + 2 - 1 J > 0. 

Thus u is not a constrained viscosity solution on [0, 00). 

Now we consider the case when c\ + C2 < 0. Let <f>(x) = (ci + C2) + (|ci — C2I + 2)x + 
(ci + C2 — l)x 2 . Then we can verify (u — 4>){x) < (u — 0)(O) = for x small and <p'(x) = 
\ c i — c 2 | + 2 + 2(ci + c 2 — l)x and 0"(x) = 2(ci + c 2 — 1). Then we compute 

min {0(0) - 0"(O), 0'(O) - 1} = min { Cl + c 2 - 2(c x + c 2 ) + 2, | Cl - c 2 | + 2 - 1} 

= min {2 - (c x + c 2 ), |ci - c 2 | + 1} > 0; 

which again demonstrates that u is not a constrained viscosity solution of (14. 2 j) on [0, 00). 

One observes that any linear combination of x+c and Cie a: +C2e _:r can not be a constrained 
viscosity solution of (14. 2 j) on [0, 00) either. In addition, functions of the form u{x) — (x + 
c)I{x>a} + [c\t x + C2e~ x )I{ x < a } are not constrained viscosity solution of (I4.2jl on [0, 00), where 
a, ci, C2 are appropriately selected constants so that u G C 1 ([0, 00)) H C 2 ([0, 00) — {a}) and 
solves (H2D in (0,oo). 

Finally we note that for the corresponding controlled process X(t) = x + \/2W(t) and 
the reward functional K x J °° e~ t dZ{t), A® = 0. The reason is that the Brownian motion 
W, starting from 0, changes sign infinitely many times and hence can not satisfy the state 
constraint in any time interval [0, e]. □ 

Example 4.4. In this example, we consider the system of coupled QVIs 

min {ru(x, a) — fx a xu'(x, a) cr 2 x 2 u"(x, a) — X a u(x, a) + X a u(x, 3 — a), 

1 ' 2 f , (4-3) 

u'(x, a) — 1 > = 0, x G (0, 00), a G {1, 2} , 

where for a = 1,2, fi a ,a a , and X a > are constants. Moreover, we assume fii,^ satisfy 
[i\ < r < n 2 < rAl+ r+Ai-^i +A2 ' > • ^ ne can ver ify tnat tne unique solution to (14. 3 j) in (0, 00) xM 
is 

A 2 

u(a;, 1) = x, u(x,2) = x, x > 0. (4.4) 

A 2 + r - /i 2 

Moreover, one can easily verify that u(-, a), a = 1, 2 satisfy the subsolution property at the 
point x — 0. Therefore u is the unique constrained solution on [0, 00) x {1, 2}. 

The corresponding controlled dynamic is given by the regime-switching geometric Brow- 
nian motion: 

dX{t) = /i a(t) X(t)dt + a a{t) X{t)dW{t) - dZ(t), 
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where {a(t),t > 0} is a two-state continuous-time Markov chain with generator 



-Ai Xi N 

A2 — A2, 

The objective is maximize the reward J(x, a, Z) = E X)Q , J Q °° e~ rt dZ(t). Observe th at A X)Ql 7^ 



for all (x,a) G [0, 00) x {1,2}. Moreover, as demonstrated in ISong et al.l (120111 ). the value 



function V(x, a) = u(x, a) for all (x, a) G [0, 00) x {1, 2}, where u is defined in (I4.4p . □ 
Now let's present the main result of this section. 

Theorem 4.5. Assume A x , a 7^ and that the value function V(-,a) is finite for each 
{x, a) G S x Ai. Then V(x) = {V(x, 1), . . . , V(x, m))' is a constrained viscosity solution of 
(l2~TTj) on S x Ai. 

The proof of Theorem 14. 51 is accomplished by the combination of Propositions 14. 61 and 14. 71 
Proposition 14.61 shows that V is a viscosity super solution, while Proposition 14.71 establishes 
that V is viscosity subsolution. 

Proposition 4.6. The function V is a viscosity super solution of (12. lip in S x Ai. That is, 
for any (xo,ao) G S x Ai and any C 2 function </>(-, ■) satisfying 4>(xo,ao) = V*(xo,ao) and 
that cj)(x, a) < V*(x, a) for all x in a neighborhood of xo and each a £ Ai, we have 

min <^ (r - C)<j>(x Q , a ), min {D Xi <j)(xo, a ) - fi{x , a )}> < 0. (4.5) 

I i=l,...,n \ 

Proof. By the definition of V*(x , a ), there exists a sequence {x m } C such that 

x m — > xo, and V(x m , aco) — > V*(xo, aco), as m — > 00. (4.6) 

This, together with the continuity of 0, implies that 

7 m := V(x m , a ) - 4>{x m , a ) -^0, asm-^oo. 

Let B £ (xq) := {1 6 R™ : \x — xo\ < e}, where e > is sufficiently small so that (i) B £ (xo) C S 
and (ii) <p(x,a) < V*{x,a) for all (x,a) G B £ (xq) x Ai, where B £ (xq) = {x G lR n : |x| < e} 
denotes the closure of B £ (xq). Choose Z such that Z(0—) = and Z(t) = r\ for all t > 0, 
where < \r]\ < e/2. Then thanks to (14.6p . Z G A Xm! a for m sufficiently large. Let 
X(-) = X Xm,a °(-; Z) be the corresponding controlled process with initial condition (x m ,ao) 
and control strategy Z(-). Put 

9 m :=mi{t>0:X(t)<£B E (x )}. 

Let {h m } be a strictly positive sequence such that 

h m — > and j^- — > as m — > 00. 
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Note that the chosen control strategy Z guarantees that X(-) has at most one jump at 
t = and remains continuous on (0, 9 m ]. This, together with the choice of e, implies that 
X(t) G B £ (xq) for all < t < 8 m . Since </> < V* < V, we can apply the dynamic programming 
principle (13. 6p to obtain 



V(x m ,a ) > E 



m /\hm 



e- rs f(X(s-),a(s-))-dZ(s) 

+ E [e- r ^ M ™)0(X(0 m A /i m ), «(^ m A /i m ))] . 
On the other hand, Ito's formula yields 



e~ rs {r - £)<f)(X(s),a(s))ds 
e- rs D<j){X{s),a{s))-dZ c {s) 



_0<s<6 m Ah 

m 

where in the above, we have used the fact that 



E [e~ 


r(6» m A/t m )0 




" pd m Ah m 


+ E 






Jo 




r6 m /\h m 


+ E 






Jo 


-E 


E 



E 



7 m Ah m 



e- rs D4>{X{s), a{s)) ■ a(X(s), a(s))dW(s) 



0. 



A combination of ( 14.71) and ( 14.81) yields 



> E 
— E 
+ E 



Ahm 



e~ rs (f(X(s-), a(s-)) ■ dZ(s) - (r - £)</>(X(s), a(s))ds) 

Ah m 

e- rs D(j)(X(s),a(s))-dZ c (s) 
e- rs [<p(X(s), a(s-)) - <P(X(s-), a(s-))] 



_0<s<6 m Ah 

Now let rj = 0, i.e., Z(t) = for any t > 0. Then ( 14. 9 p can be rewritten as 

1 



m J 



'111 Ahm 



1 

n. 



in JO 



e~ rs (r - £)<t>(X(s),a(s))ds 



(r - £)<j)(X(s), a(s))I {s <e m} ds. 



(4.7) 



(4.8) 



(4.9) 



Note that as m — > oo and hence h m — > 0, by the right continuity of the trajectory (X(s), at(s)), 
e~ rs {r — C)(f>(X(s), a(s))/{ s <6) m } — > (r — £)0(x o ,a o ) a.s. for s G [0,/t m ]. Thus by virtue of 
the bounded convergence theorem, we have 



(r - £)(fr(x ,a ) > 0. 



(4.10) 
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On the other hand, if we choose r\ = rjiei with < rji < e/2 and e« = (0, . . . , 1, . . . , 0)' 
being the ith unit vector, i = 1, . . . , n, then (14. 9 j) reduces to 



Ah m 

7m > -E / e""(r - £)</>(X(s), a(s))ds + /»(a; m , a )^ + 0(x m - r?, a ) - a ). 



o 



Now sending m — >■ oo, we have 

fi(x , a )r)i + 0(x o - n, a ) - 0(x o , a ) < 0. 
Finally, dividing the above inequality by rji and letting r\i — > lead to 

D x .(f)(x , a ) - fi(x ,a ) > 0, i = l,...,n. (4.11) 
Now ( 14.5ft follows from a combination of ( 14. 101) and ( 14. lip . □ 

Proposition 4.7. TTie function V is a viscosity subsolution of ( 12. lip in S x Ai. That 
is, for any (xo, ao) £ S x M and any G C 2 snc/i i/iai <£>(a;o,ao) = V*(:ro,a:o) and i/iat 
<^(x, a) > V*(x, a) for x G 5 in a neighborhood of xq and each a <E Ai, we have 



min <^ (r - £)y?(x , a ), min {D Xi ip(x , a ) - /i(x , a )} > < 0. (4.12) 

I i=l,...,n \ 

Proof. Suppose on the contrary that (I4.12p was wrong, then there would exist some (xq, a ) G 
S x M, a <p G C 2 with (V* — (p)(x, a) < (V* — (p)(x , a ) = 0, and a constant A > such 
that 



min <^ (r - C)tp(x ,a ), min {£>„. y?(x , a ) - fi(x , a )} > > 2A > 0. (4.13) 

[ t=l,...,n J 

In what follows, we will derive a contradiction to (I4.13p . This is achieved in several steps. 
First we use the generalized Ito formula and ( I4.14p to obtain ( I4.15p . from which we obtain 
( I4.16P and (I4.17p . Next, detailed analysis using the monotonicity of the functions V* and / 
leads to ( I4.22p . Then we claim in ( I4.23P that the last term in (I4.22p is bounded below by 
a positive constant, from which, with the aid of dynamic programming (13.51) . we obtain a 
contradiction to (I4.13p . The final step of the proof is devoted to the proof of ( I4.23p . 

Step 1. As in the proof of Proposition 14.61 let { x m} C S be a sequence such that 

x m — > Xq, and V(x m , a ) — >■ V*(x , a ) as m — > oo, 

and 

7 m := V(x m , a ) - <f(x m , a ) -i> 0, as m ->■ oo. 

Choose m sufficiently large so that \x m — x \ < e/2. Fix some Z G A Xm>ao and let X(-) = 
X Xm,a °(-, Z) be the corresponding controlled process. Define B £ (x ) '■— {x G S : \x — x \ < e\, 
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where e > is small enough so that (i) <p(x,a) > V*(x,a) > V(x, a) for all (x,a) G 
B £ (x ) x M, and (ii) 

min < (r — C)f(x, a), min {D Xi tp(x, a) — fi(x, a)} > > A > 0, V(x,a) G -B £ (x ) X M. 
[ ' i=l,...,n ' J 

(4.14) 

Let # m := inf {t > : ^ 5 £ (xo)}. Then for any t > 0, we have from the generalized 
Ito formula that 



ip(x m ,a ) = E 
+ E 



-(^V(X(t A a{t A ra -)) + / e~ rs {r - C)<p(X(s), a(s))ds 



tA8 m - 



E 



e- rs J2D x MX(s),a(s))dZ^(s 

i=l 

e~ rs [<p{X{s), a(s-)) - <p(X{s-\ a(s-))) 



,0<s<M6» 

77 



Note that 



n 

= - X^s-^D^Xis-) + z(X(s) - X(s-)), a(s-)) 



i=l 



AZ t ( S )D XiV (X(s-) + z(X(s) - X(s-)), a(s-)) 



i=l 



for some z G [0, 1]. But by virtue of (14. 14ft . for all < s < t A 6 m , we have 

D Xi ip{X{s-) + z(X(s) - X(s-)), a(s-)) > fi(X(s-) + z(X(s) - X(s-)), a(s-)) + A. 

Further, since X(s) < X(s— ) + z(X(s)— X(s— )) < X(s— ) and that a) is non-increasing, 
we have 

MX(s-) + z(X(s) - X(s-)), a(s-)) > MX(s-), a(s-)). 
Then using (I4.14p again, we obtain 



ip(x m ,a ) > E 



ptA9 m - 

e- r{tAdm) <p(X(tA6 m -),a(tA6 m -))+ / e~ rs A(ds + 1 ■ dZ (s)) 

Jo 

tA6 m - 



+ E 



e- rs f(X(s-),a(s-))-dZ(s) 



(4.15) 

where 1 = (1, . . . , 1)'. Now letting t — )■ oo in (14.151) . it follows that on the set {9 m = oo}, we 
have 

V(x m ,a ) = ip(x m ,a ) + 



> E 



e- rs f(X(s-),a(s-))-dZ(s) 



A 

r 



(4.16) 
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Step 2. On the set {9 m < 00} , we have by letting t — > 00 in (I4.15P that 



ip(x m ,a ) > E 



+ AE 



(4.17) 



<p(X(9 m -), a{6 m -)) + / e~ rs f{X{s), a(s)) ■ dZ{s) 
e- rs {ds + t-dZ{s)) 
and X(8 m — ) G -B e (x ). Thus there exists some A G [0, 1] such 

-)-AAZ(0 m ) Gd£ £ (x )- 



Note that X(6 m ) < X(8 m 
that 

x x := X(9 m -) + X(X(9 m ) - X{6 m -)) = X 
Moreover, X(9 m ) < x x < X(d m -). Note that 



ip(X(9 m -), a(9 m -)) - (p(x x , a(9 m -)) 
= (X(9 m -) - x x ) ■ D<p{X{9 m -) + z{X{9 m ) - x x ), a{9 m -)) 
= XAZ(9 m ) ■ D<p(X(9 m -) + z(X(9 m ) - x x ), a(9 m -)). 

But f )4.14p and the monotonicity of fi(-,a) imply that 

D x .<p{X{6 m -) + z(X(9 m ) - x x ), a(9 m -)) > h{X{9 m -) + z(X(9 m ) - x x ), a(9 m -)) + A 

>fi(X(9 m -),a(9 m -))+A. 

This, together with the fact that AZi(9 m ) > 0, leads to 

<p(X(9 m -), a(9 m -)) - <p(x x , a(9 m -)) > XAZ(9 m ) ■ (f(X(9 m -), a(9 m -)) + At) . (4.18) 

Combing (14.171) and (I4.18H . we obtain 

V(x m , a ) = (f{x m , a ) + 7 m 
> E 



e~ rs f(X(s-), a(s-)) ■ dZ(s) + A / e~ rs (ds + 1 • dZ(s)) 

Jo 

+ E [\e~ re ™AZ(9 m ) ■ f(X(9 m -), a(9 m -))] 
+ E [e~ re ™ (ifi(x x , a(9 m -)) + XAAZ(9 m ) ■ 1)} + lm . 



(4.19) 

Note that x x G B £ (x ) and hence <p(x\, a(9 m — )) > V*(x\,ot{9 m —)). On the other hand, 
since X(9 m ) < x x < X(6 m —), it follows from (13. 4 p and the monotonicity of / that 



V*{x x ,a(9 m -)) > V*{X{9 m ),a{9 m -)) + (x x - X{9 m )) ■ f(x x ,a(9 m -)) 

> V*(X(9 m ),a(9 m -)) + (1 - X)AZ(9 m ) ■ f(X(9 m -),a(9 m -)). 



(4.20) 



A similar argument as that in ISong et al.l ( 2011 ) yields that 

E [e~ re ™V*(X(9 m ),a(9 m -))} = E [e~ rd ™V*(X(9 m ),a(9 m ))} . 
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(4.21) 



Now put (T4T20D and (I42TD into (T4TT9]) and we obtain 



V(x m ,a ) > E 



e- TS f(X(s-) } a{s-)) ■ dZ(s) + e- re ™V*(X(6 m ), a(9 m )) 



+ AE 



e~ rs (ds + 1- dZ{s)) 



LJQ 



+ 7r, 



+ (1 - A)E [e- rf> ™AZ{e m ) ■ f{X{6 m -),a{e m -))] 
+ AE [e- r0 ™AZ(6 m ) ■ (f(X(9 m -), a(9 m -)) + Al)] 

e- rs f(X(s-),a(s-)) ■ dZ(s) + e' r9 -V*(X(6 m ),a(e m )) 



= E 

LJO 

+ AE 



e~ rs (ds + 1- dZ{s)) + Ae- re '"l • AZ(6. 



LJQ 



+ 7m- 



(4.22) 



We now claim that for some constant k > that does not depend on m, we have 



E 



e- rs (ds + 1- dZ(s)) + \e- r9m l ■ AZ(6 m ) 



> K. 



(4.23) 



Step 3. Assume (14. 23ft for the moment. Then (14.22ft can be rewritten as 



V(x m ,a ) > E 



e-™f(X(s-), a{s-)) ■ dZ(s) + e- rd ™V*{X{6 m ), a(8 m )) 



Ak + 7 r , 



(4.24) 

Combining (I4.16P and (14.241) . and then taking supremum over Z e A Xmtao , it follows from 
the weak dynamic programming principle (13.51) that 

V(x m , a ) > V(x m , a ) H A Ak + 7 m > V(x m , a ), 

r 

for m sufficiently large. This is a contradiction. So we must have ( I4.12p and hence V is a 
viscosity subsolution of (12. lip . 

Step 4- Now it remains to show (14.231) . To this end, we consider the function W(x, a) : = 
| X — Xq J — for (x,a) € B £ (x ) x M.. Then it follows that 



(£ - r)W(x, a) = 2(x - x ) ■ b(x, a) + -tr(2Ia(x, a)a'(x, a)) - r(\ 



|2 2^ 

X — Xq\ —6 



Since W, b, and a are continuous, and M. is a finite set, it is obvious that 

\{C-r)W{x,a)\ <K<oo 

for some positive constant K. Now let K := 2g * and define W(:c, a) = ir VK(x, a) for 
(x, a) G B £ (xq) x .M. Then it follows immediately that 



|(£ -r)W(x,a)\ < 1, G B £ (x ) xM. 



(4.25) 
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Moreover, we have 

D x W(x, a) = 2K {x - x ) ■ e { > -1, (4.26) 

where = (0, . . . , 1, . . . , 0)' denotes the zth unit vector. Let x m , 9 m , Z G A XmtOl0 etc. as 
before. Using (14. 25ft . (I4.26j) . and generalized Ito's formula, detailed computations similar to 
those in Step 1 yield 



E [e- rd -W(X(8 m -),a(d m -))} - W(x m ,a ) < E 



e~ rs (ds + 1- dZ{s)) 



(4.27) 



Also, recall that X(6 m ) < x x < X(6 m -). It follows from (T4T26]) that 
W(X(6 m -), a(9 m -)) - W(x x , a(6 m -)) 

n 

= D Xi {x x + z(X(9 m -) - x x ), a{6 m -))(X(6 m -) - x x ) ■ e { 



x D ^x + 4X(6 m -) - x x ), a{e m -))^Zi{e n 

) = -Al ■ AZ(6 m ). 

1=1 

Combining (I4.27P and (I4.28p . we have 



(4.28) 



n 

> -A^AZ, 



E 



e~ rs (ds + 1- dZ(s)) + Ae" rem l • AZ(9 r , 



> E [e- rd ™W(x x ,a(e m -))]-W(x m ,a ). 



But x x e dB e (xo), and consequently W(x x , a{6 m — )) = 0. Also, it is immediate that 
W(x m ,a ) = K (\x m — x \ 2 — e 2 ) < K Q ((^) 2 — e 2 ) = — \K Q e 2 . Hence it follows that 



E 



e~ rs (ds + 1- dZ(s)) + Ae _r " m l ■ AZ(9 m ) 



> -K e 2 =:k>0. 



This establishes (14.231) and hence finishes the proof of the theorem. 



□ 



5 Viscosity Solution: Uniqueness 

Our goal is to establish a strong comparison result for constrained viscosity solutions of 
(12. lip . To this end, we need the following lemma. 

Lemma 5.1. Let s(x) = x ■ 1 = Y^i=i Xi an< ^ 

u(x, a) := e~ Xs{ - x) u(x, a), v(x, a) := e~ Xs ^v(x, a), V(x, a) E S x M } (5.1) 

where A > 0. Then 
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(5.2) 



(a) u(x, a) is viscosity subsolution of (12. lip if and only ifu(x,a) is a viscosity subsolution 
of 

min {ru(x, a) — H\(x, a, u(x, a), Du(x, a), D 2 u(x, a)) — Qu(x, -)(a), 
min {e Xs{x) [Xu(x,a) + D Xt u(x,a)] - fi{x,a)\ } = 0, 

i=l,...,n 

where for any (x, a, q,p, A) G R n x M x R x E" x S n , 

H\(x, a, g,p, A) =-tr(acr / (x, a) A) + — (t'aa'(x, a)p + p'aa'(x, 

l2 



+ b(x, a) • p + Xqb(x, a) ■ 1 + — g [cr (x, a)l| 



and 



Qm(x, -)(a) = ^gai-u(^,j) = ^g ai [u(a;,j) - u(x,a)). 

3=1 3=1 

(b) Similarly, v(x,a) is viscosity supersolution of (12. lip if and only ifv(x,a) is a viscosity 
supersolution of (15. 2p . 

Proof. We prove part (a) only; the proof of part (b) is similar. Suppose u is viscosity 
subsolution of (12. lip . Let </?(•, a) G C 2 , a e M. and let (x , a ) be a maximum point of u — (p 
with (w — v3)(xo, a o) = 0. Put 

(p(x, a) := e Xs(x) (p(x, a). 

Then it is easy to verify that <£>(■, a) G C 2 , a & A4 and (a;o, ao) is a maximum point of u — <p 
with (m — <p)(x , a ) = 0. Since u is viscosity subsolution of (12. lip , we obtain 

r^(xo,«o)--tr(CTCTXxo,a )^V(^o,ao))-&(^o,ao)-^V ? (^o,ao)-^^ojV ? (a ; o, j) < 0, (5.3) 
or 

min {^^(xcao) - /i(^o,ao)} < 0. (5.4) 

i=l, ...,n 

Since <p(x,a) = e Xs ^0(x,a), we compute 

D x .(p(x,a) = e Xs{x) [\ip(x,a) + D Xi <p(x, a)], 

and 

D XiX .<f(x, a) =e Xs{x) [X 2 (p{x, a) + \(D Xj <p(x, a) + D Xi <p(x, a)) + D XiXj <p(x, a 
In other words, 

D(p(x, a) = e Xs{x) [\tp(x, a)l + D<p(x, a)] , 
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and 

D 2 ip(x, a) = e Xs(x) [\ 2 <p{x, a) 11' + \{lDip{x, a)' + Dfi(x, a)l') + D 2 (p{x, a)] . 
Then substituting Dip and D 2 ip into (15. 3p leads to 

m 

> r^(x ,a ) - ^2q ao j&{xa,j) ~ b(x ,a ) ■ {X0{x o ,a o )l + D0(x o ,a o )) 

3=1 

- -tr (cra'(x , a ) [\ 2 <p{x , a )ll' + \(lD<p(x , a )' + ^^(so, a )l') + D 2 <p(x Q , a )] ) , 
which can be rewritten as 

r0(x o , a ) - #a(zo, «o, <^(^o, «o), D0(x o , a ), D 2 <p(x , a )) - Q<^(x , -)(ao) < 0. (5.5) 
Similarly, ( 15. 4p can be rewritten as 

min {e As(xo) [A^(xo,a ) + D Xi <p{x ,a )] - fi{x ,a )} < 0. (5.6) 

i=l,...,n 

Therefore in view of (15. 5p and (15.61) . u is a viscosity subsolution of (15. 2p . 

Conversely, let u be a viscosity subsolution of ( 15. 2ft . Recall a) = e As( - :E - ) 'u(x, a). Let 
</?(•, a) G C 2 ,a G M. and (xo,«o) be a maximum point of u — <p with (u — <p)(xo,aca) = 0. 
Put <p(x, a) := e~ Xs ^(p(x, a). Detailed calculations as above show that u is a viscosity 
subsolution of (l2TTTj) . □ 

Lemma 5.2. For even/ £ G 5, t/iere ern'st 77 = ry(£) el™ and a = a(£) > suc/i t/iat 

£? ta (x + tr/) C S, Vx G 5 D B a (£), Vt G (0, 1], 

where B a (x) = {y G M. n : |y — x| < a}. 



Proof. See lAtar and Budhirajal ( 120061 ) . □ 

With Lemmas 15.11 and 15.21 at our hands, we are now ready to establish the strong com- 
parison result for the constrained viscosity solution of ( 12. lip . 

Theorem 5.3. Let u G USC(Sx M; R m ) andv G LSC(Sx M; M. m ) be respectively viscosity 
subsolution on S x M. and supersolution inSxM of (12. lip and satisfy 

\u(x, a)\ + \v(x, a)\ < K(l + \x\ p ), V(x, a) G S x M, (5.7) 

where K and p are positive constants. Assume that for some positive constant kq, we have 

\b(x, a) — b(y, a)\ + \cr(x, a) — a(y, a)\ < k \x — y\ , (5.8) 

b(x,a)'l < no and \a(x,a)'l\ < k , (5.9) 
for all (x, a) G S x M.. Then we have 

u(x,a) < v(x,a), W(x,a) G S x M.. 
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Proof. Suppose on the contrary that 



M := max sup [w( x, a) — v(x, a)] > 0. 



(5.10) 



We will derive a contradiction in the following. Define u and v as in (|5.ip . where A > is a 
constant to be determined later. Thanks to (|5.7p . u and v are uniformly bounded. Moreover, 
we have 

lim + = 0, Vet G .M. 

>oo, x£S 

Therefore in view of (I5.10P and the facts that M. is finite and that u — v is upper semicon- 
tinuous, there exist some bounded set O of S and (x,£) e O x M, such that 

M := maxsup[£t(x, a) — v(x, a)} = max[tt(x, £) — v(x, £)] = u(x, £) — v(x, £) > 0. (5.11) 



Let r] = i](x) be as in Lemma 15.21 For any e, 5 G (0, 1) and (x, y) G O x O, define 
$(x, y) = $e,sAx, y) ■= u(x, £) - v(y, £) - <p(x, y), 

2 



{x,y) :-- 



(y-x) - 5r] 



+ Six — x\ . 



(5.12) 



Note that $ is USC and hence achieves its maximum M = M e ^ t \ on the compact set O 2 at 
(x, y) := (x £j s,\,y £ ,8,\)- By virtue of Lemma |5T2| x + e5r] G S°. Also, since 



${x,y) = u{x,£) - v(y,£) 



-{y -x)-5r] 



— 5 \x — x\ 



> $(x, x + e5r]) = u(x, £) — v(x + eSr], 



we have 



u(x, £) — v(y, £) — u(x, £) + v(x + e5r], £) > 



-{y -x)-6r) 



+ Six — x\ . 



Multiplying e 2 on both sides of the above equation, we see that for each 5 and A, x — y — > 
as e — > 0. Further, by virtue of (15.111) . we have 



lim sup 



~(y -x)-5rj 
e 



+ 5 \x — x\ < 0; 



and therefore 



x x, and —{y — x) —> 6r), as e — > 0. 
In particular, it follows that 

y = x + e5r] + o(e) = x + e5r] + o(e), as e — > 0, 



(5.13) 



(5.14) 
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and hence y G S° for e sufficiently small. 

The function x h-> u(x,£) — <p\{x) achieves its maximum at x, where 



<f>i( x ) = v (y^) + 



: (y -x)-6rj 



+ Six — x\ 



Moreover, we compute 



2 (\ \ 2 

■(y - x) - Si]) + 2S(x - x), and D 2 (j)i{x) = —I + 251. 



e \e 



Hence it follows from Lemma 15. 1[ the definition of viscosity subsolution, and Ishii's lemma 
that for some M e S n , (—D(fri(x), M) e P 2 ' + u(x,£), such that 



Thus either 



min |r-u(x, £) — H\(x, £, u(x, £), D(f>i(x), M) — Qu(x, •)(£), 

min {e Xs(i) [\u(x,£) + D^ii) ■ a] - fi(x,£) \ } < 0. 

i=l,...,n J 

min \e Xs{£} [\u(x,£) + Dfaix) ■ ei ] - fi(x } £)\ < 0, 

i=l,...,n 



(5.15) 



or 



ru{x,£) - H x (x,l,u(x,?),D<l>i(x),M) -Qu{x,-){£) < 0. (5.16) 
On the other hand, the function y i-> v(y,£) — 4> 2 {y) achieves its minimum at y, where 



<fa(y) = u(x,£) - 



: (y -x)-5rj 



S \x — x\ 



Direct calculations reveal that 



DMv) 



2 (\ 



e \ e 



(y - x) - Sr]) , and D 2 (p 2 (y £ ) = — ~ L 



Hence the definition of supersolution and Ishii's lemma imply that for some N E S n , we have 
(D<p 2 (y),N)eP 2 >-d(y,£) and 



min \ rv(y, £) - H x (y, £, v(y, £), D<p 2 {y), N) - Qv(y, ■)(£), 

min n {e Xs ^[Xv(y,£) + D^y) ■ ei ] - fi(y,£) \ !> > U. 



(5.17) 



Case 1. Now suppose (15.151) is true. Recall u(x,a) = e Xs ^u(x,a) and v(x,at) = 
e~ Xs ^v(x,a). Then we have from (15. 15ft and (15. 17ft that 

> min {e Xs ^[Xu(x,£) + Dfaffl ■ e % \ - - e Xs ^[Xv(y,£) + D<j> 2 {y) ■ e,,] + f^yJ)} 

i=l,...,n 

= min {X(u(x,£) - v(x,£)) + (e^D^x) - e Xs ^ Dcj> 2 (y)) ■ e, - (fi(x,£) - f t (y,£))} . 
i=i,.. .,n 
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Hence it follows that 

X(u(x,£)-v(x,£)) 

< max {(Mx,£) - mi)) - (e^Dfcix) - e Xs ®D<j> 2 {y)) • e,} 

1=1, ...,n 

2 /l 



max { (fi(xj) - fi(y,£)) - e* 

«=l,...,n 



3 As(x) 



(y-x)-Srj )+26(x-x)) ( 5 - 18 ) 



+ e 



MS) 



«rt2 /l 



£ \ £ 



-(j/ - 5) - 5?7 



Thanks to (15. 13j) . ( 15. 14)) . and the continuity of /j, the right-hand-side of ( 15.18)) converges 
to as e j. 0. Note that for any x G O, we have 



u(x, £) — v(x, ■ 



<&(x,x) < $(x,y) 

u(x, I) - v(y, £) - (j)(x, y) < u(x, £) - v(y, £). 



In particular, taking x = x leads to u(x,£) — v(x,£) < 0, which gives a contradiction to 
(15. lip . Hence, Case 1 is impossible. 

Case 2. Now suppose (15. 16)) is true. Then it follows from (15 . 16)) and ( 15.17)) that 



r(u(x,£)-v(y,£)) - [Qu(x, ■)(£)- Qv(y, ■)(£)} 

- [H x (x,£,u{xJ),DMx),M) - H x (y,£,v(y e ,£),D<f> 2 {y),N)} < 0- 

Using the definition of H\, 

H x {x, £, u(x, £), D<j> s {x), M) - H x (y, £, v(y, £), D<f> 2 (y), N) 



(5.19) 



(tr(£7<r'(x, £)M) - ti(aa'(y, £)N)) + A {u{x, £)b{x, I) - v(y, £)b(y, £)) ■ 1 



+ - ([lW(i, i)D(f> 1 (x) + W(z,*)l] - [l , aa , (y 1 £)D<p 2 (y) + D0 2 (y)W(y,*)l]) 

+b(x,£) ■ Dfa{x) ~ KM ■ D<t> 2 {y) + y (u(x,£) \a'(x,£)l\ 2 - v(y,£) \a'(y,£)l\ 2 
Hence it follows that 



\2 

r - Xb(x,£)'l - — \a'{x,£)l\ 



{u(x,£)-v(y,£)) 



< [Qu(x, ■){£) - Qv{y, ■){£)} + Xv(y,£)(b{x,£) - b(y,£)) ■ 1 
+ ^[tv(aa'(x,£)M) - tr(aa'(y,£)N)} 

+ ^ ([t'aa'(x,£)DMx) + £tyi(z)W(z,*)l] - [lW(y, £)Dfa{y) + D<t> 2 {y)'aa'{y,£)l\) 

+ b(x,£) ■ D(j)i(x) - b{y,£) ■ D(j> 2 {y) + ^v{y,£) (\<r'(x,£)l\ 2 - \a'(y,£)l\ 2 

(5.20) 
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Using ( 15. 9p and the fact that r > 0, we choose A > sufficiently small so that 

\2 

r - Xb{x,£)'l - — \a'{x,£)l\ 2 > 0. 

Next we analyze the terms on the right-hand side of (I5.20p . Recall that 

y) = u(cc, £) — v(y, £) — <p(x, y) > x) = u(x, £) — v(x, £) — 5 2 \r)\ 2 

Note also qu < 0. Thus it follows that 

q u [u(x, £) - v(y, £)] < q u [u(x, £) - v(x, £) + <f)(x, y) - S 2 \r]\ 2 ] . 

This, together with floTTTD . (15TT3|) . (15TT4|) . and the fact that u - v is USC, lead to 
limsup [Qu{x,-){£)-Qv{y, ■){£)} 

= limsup ^2q ej (u(x,j) - v(y, j)) + q u (u(x,£) -v(y,£)) 



e-i.0 



< ^2qej(u(x,j) - v(x,j)) + q u (u(x,£) - v(x,£)) - 5 2 q u \r)\ 2 

qij(u(x, £) - v(x, £)) + q u (u(x, £) - v(x, £)) - 5 q u \r]\ = -5 <?« \v\ ■ 
By virtue of Ishii's lemma, 



lim - (tr(aa'(x £ , £)M) - tr(aa'(y E , £)N)) = 0. 

£4,0 Z 

Next, using ( 15. 8 j) and (15. 13j) . and noting that v is bounded, we have 



lim 

£4.0 



Xd(y,£)(b(x,£) - b(y,£)) ■ I + ^v(y,£) [ \*'(x,£)l\* - \a'(y,£)l\' 



0. 



Similarly ( 15 .8p and (I5.13P imply that 

[l'a</(x,i)D(f> 1 (x) + D<l> 1 (x)'<Ta , (x,£)l) - [l'*o'(y,£)D<i> 2 (y) + D<f> 2 (y)'aff'(y,t)l) 
= At (^'(^ t) - aa'{y, £)) ■ {^{y - x) - 5rj 
2 (\ 



+ 25 (lW(z, - z) + (5 - x) W(x, 
—7-0, as e — > 0; 



and 



b{x,£) ■ Dfa(x) -b{y,£)D<t> 2 {y) 

= ~(b(x, £) - b(jj, £)) ■ (^(y - x) - 6rj ) + 25b(x, £) ■ (x - x) 
-> 0, as e ->■ 0. 
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Now letting e I and using (I5.21l) - (l5.26p in (15.201) . we conclude that for sufficiently small 

A, 

limsup u(x, £) — v(y, £) < —5 2 qu Ivf • 
But 5 > can be arbitrarily small, as argued in Case 1, it follows that 

u(x, £) — v(x, t) < u(x £ , £) — v(y £ , £) — > 0, as e — > and S — > 0, 

which again contradicts ( 15. lip . Therefore for any x G S and a 6 M, we have u(x, a) < 
v(x,a), as desired. □ 

Remark 5.4. Note that under condition (15. 9p . the value function is bounded above by an 
affine function. In fact, for any Z G A Xj0l with (x, a) G S x A4, we have 

d(e- r *X(t)) = e~ rt [(b(X(t), a{t)) - rX{t))dt + a(X(t), a(t))dW(t) - dZ{t)} . 

Thus 

e- rt dZ(t) = e- rt {b{X(t), a(t)) - rX(t))dt + e- H a(X(t), a{t))dW(t) - d{e- rt X{t)), 
from which it follows that 

poo /*oo 

/ e- rt t ■ dZ(t) < / e- rt t ■ [(b(X(t), a(t)) - rX(t))dt + a(X(t), a(t))dW(t)} + 1 
Jo Jo 

Taking expectations on both sides and using ( I5.9p . we have 



x. 



E 



/ e~ rt l ■ dZ(t) < E f e~ rt (K dt + 1 • a(X(t), a(t))dW(t)) + l-x = — + t-x. 
Jo Jo r 



In the above, E J °° e ri l • a(X(t), a{t))dW{t) = since 1 • a is uniformly bounded. Hence, 
it follows that 

V(x,a)= sup E f°° e~ rt f(X(t—), a(t—) ■ dZ{t) < \\f\\oo (— + 1 ■ x) . 

zeA x , a Jo V r / 

Finally we summarize the main result of this paper from Theorems 14.51 and 15.31 

Theorem 5.5. Assume (I5.8p . (15. 9p . and that A XtQl ^ for every (x, a) G S x Ai. Then the 
value function V defined in ( 12. 6 p is the unique constrained viscosity solution of the system 
of coupled quasi-variational inequalities ( 12. lip on S x M.. 

Remark 5.6. At first look, condition ( 15.91) seems rather restrictive. Simple models such 
as regime-switching geometric Brownian motion considered in Example 14.41 are excluded. 
However, the following example indicates that in general, one can not remove (I5.9P ; otherwise, 
uniqueness may not hold. 
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Example 5.7. Let's consider a 1-dimensional squared Bessel process subject to control 



dX(t) = dt + 2^\X{t)\dW{t) - dZ{t), 



(5.27) 



with reward functiona l J(x,Z) = K x / °° e t dZ{t), where x > 0. It is well known (see, e.g., 
Revuz and Yorl (11999l )) that the stochastic differential equation 



^ t ) = x + t + 2 [ y/\£$)\dW(s) 
Jo 



has a unique strong solu tion £ x , and for all t > 0, £, x (t) = x + \W(t)\ 2 > if x > 0. Moreover, 



Song et al.1 fl2012f ). it follows that P {r = 0} = 1, where r := inf {t > : f °(t) > 0}. 



using 

Hence it follows that A x ^ for all x G [0, oo) 
The corresponding QVI is 



min {u(x) — u'{x) — 2xu"(x), u\x) — 1} = 0, x G (0, oo). 



(5.2? 



One can easily check that v(x) = x + 1 is a constrained viscosity solution to f!5.28j) on [0, oo). 
In fact, for x > 0, 

min {f (x) — v'(x) — 2xv "(x), v '(x) — 1} = min {x + 1 — 1 — 2x • 0, 1 — 1} = min {x, 0} = 0. 

Moreover, the subsolution property holds at the point x = since for any G C 2 with 
(f — 4>){x) < (v — 0)(O) = 0, we have </>'(0) > 1 and hence 

min {0(0) - 0'(O) - 2 • • 0"(O), 0'(O) - 1} = min {1 - 0'(O), 0'(O) - 1} < 0. 



Thus v(x) = x + 1 is indeed a constrained viscosity solution to (15.281) on [0, oo). 

Next, we demonstrate that (15.281) has at least another constrained viscosity solution on 
[0,oo). First we note that the function ip(x) := sinh(A/2x) is increasing and solves the 
equation u(x) — u'(x) — 2xu"(x) = in (0, oo). Further, straightforward calculations reveal 
that 

1 



, cosh(V2x) „ 

^ ( x ) = — h= — ' V> w 



2v/2 



cosh(y / 2x) ^ v2 sinh(v / 2lr) 



3/2 



cosh(v2~c) 



The equation ^"(x) = or equivalently g . (1| — 
by z. Now we claim that the function defined by 



2x has a unique positive root, denoted 



u[x) 



sinh(v 2x) 



'2z 



cosh(v2i 



-I(o,z]( x ) + \ x — z + sinh(v / 2 _ i) 



cosh(V2i) 



is the only constrained viscosity solution to (I5.28P on [0, oo). In fact, one can directly verify 
that u(x) is a solution to (I5.28P for x > 0. As in Example I4.2[ it remains to verify the 
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subsolution property at the point x = 0. To this end, let G C 2 with (u — 4>)(x) < 
(u — 0)(O) = for x G [0, oo) in a neighborhood of 0. Then 0(0) = and for x > 0, 
4>(x) > u(x) > 0. Thus we must have <f)'{0) > and hence 

min {0(0) - 0'(O) - • 0"(O), 0'(O) - 1} < 0. 

The desired conclusion follows. Note that u also satisfies the polynomial growth condition 

m- 

In terms of the singular control problem ( 15.271) . it turns out that the value function 
V(x) = v(x) = x + 1. In fact, from the state constraint, we have Z(t) < x + W(t) 2 for any 
t > 0. Therefore 

/»oo /»oo /»oo /»oo rs 

J{x, Z) = E x / e~ l dZ{t) = E X / e~ s dsdZ{t) = E X / dZ{t)e~ s ds 
Jo Jo Jt Jo Jo 

/*oo /»oo 

<E X e' s (x + W 2 (s))ds = e~ s (x + s)ds = x + 1. 
Jo Jo 

Furthermore, it is easy to check that the control Z*(t) = x + W 2 (t) is optimal and J(x, Z*) = 

x + Hence V(x) = x + 1 as claimed. □ 

We finish the section with a hierarchical PDE characterization for the boundary behavior 
of the solution to (12. lip . Let i C {l,2,...,n} be an index subset. For a vector v = 
(vi,...v n ) G M n , we induce a smaller vector v l := (vi) ie £ G i.e. vf = for all 

i G {1, . . . , n}. Typically, this notation will be used for v — b,a, f, £, X, Z. 

In a reverse direction, for a vector v G R w , we define a larger vector v e G M. n by 



«i, if j = li, 
0, otherwise. 



For a function g : R™ x M h> R™, we induce another function / : l f x M 4 I™ such that 
g-(x,a) = g £i (x- e } a). 

The following assumption is imposed. 

(HI) bi(x, a) = <Ji(x, a) = on {x G R+ | Xi = 0}. 

This basically means that, in the content of ecosystem modeling, once the ith species becomes 
extinct, it will never revive, i.e. if (Q) t = for some t, then (d) s = for all s > t. 

Thanks to (HI), (12. ip implies following sub-dynamics: 

d( e (t) = b\C l {t), a(t))dt + a £ (( e (t),a(t))dW e (t), ( l (0) = x" a(0) = a. (5.29) 

Therefore, we can look at following subsystem. Suppose the survived species are indexed by 
t with its remaning amount x G R^', then the associated value function can be defined as 

/>oo 

J\x, a, Z) := E / e~ rs f (X x ' a ' e (s-), a(s-)) ■ dZ\s) 
Jo 
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and 



V(x,a) = sup J(x,a,Z). 

Z £ -4-a; , a 



For all x G K 1 * 1 , £ G M m , p G A G S^, we define a function 

G e {x,a,£,p,A) = min{r-£ a - -tr{a\a l )'{x l ,a)A) -b\x e ,a) •p-J^^Cj, 

j'=i 

min { Pi - //(a, a)}} 
»=i,-,KI 

Then, one can apply induction to the previous results to show that, V(x) = (V(x, ot)) £ 
is the unique solution of 



G(x, a, V(z), a), L> 2 y(x, a)) = 0, (x, a) G x , * 

= (x,a) Gf^ 1 x M,\£\ =n-l. { ' 

6 Conclusions and Remarks 

In this work, we considered a class of singular control problems with state constraints and 
regime-switching. The controlled dynamic is given by a regime- switching diffusion confined 
in the unbounded domain S = and the objective is to maximize the total expected 
discounted rewards from exerting the singular control. Using the weak dynamic programming 
principle, we showed that the value function is the unique constrained viscosity solution of 
the system of coupled nonlinear quasi- variational inequalities (12. lip . 

Throughout our analysis, the discount rate r was fixed. It is interesting to ask how the 
solution, with appropriate scaling of the cost, will behave as r — > 0; and how the limit, if 
it exists, relates to that of the average cost control problem. A number of other questions 
deserve further investigations. In particular, it is worth studying the case when the random 
environment or the Markov chain a is unobservable. 
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